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DETAILED ACTION 

Continued Examination Under 37 CFR 1.114 

1 . A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 
CFR 1.17(e), was filed in this application after final rejection. Since this application is 
eligible for continued examination under 37 CFR 1.1 14, and the fee set forth in 37 CFR 

1 .17(e) has been timely paid, the finality of the previous Office action has been withdrawn 
pursuant to 37 CFR 1.1 14. Applicant's submission filed on 10/23/2008 has been entered. 

Response to Amendment 

2. The amendments, filed on 10/23/2008, have been entered and made of record. Claims 1 and 6 
have been amended. Claims 1-4, 6-11, 13-15, and 17-32 are pending. 

Response to Argument 

3. Applicant's arguments with respect to independent claims 1-4, 6-11, 13-15, and 17-32 have 
been considered but are moot in view of the newly found prior art references. 

Claim Rejections - 35 USC §103 

4. The text of those sections of Title 35, U.S. Code not included in this action can be found in a 
prior Office action. 
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5. Claims I, 3-4, 6-11, 13-15 and 17-32 are under 35 U.S.C. § 103(a) as being unpatentable 
over Gvili, "Depth Keying" SPIE Vol. 5006 (2003) in view of Podoleanu, US 6,769,769, and 
further in view of Kawaguchi, US 6,473,516. 

Regarding claim 1, Gvili discloses a method for differentiating between foreground 
objects and background objects within a scene being captured through an image capture 
device (a novel depth video camera), comprising: 

- emitting a ray of light from a light source toward an object of the scene (generating a 
light wall, Para 3.1, pp 566-567); 

- opening an aperture cover allowing access to a sensor of the image capture device for 
reflected light from the light source (deploying a fast image shutter in front of the 
CCD chip, page 567, lines 3-4); 

closing the aperture cover after a set time (the shutter is precisely controlling the 
exposure time of the CCD, page 567, lines 1 7- 1 8), the predefined amount of time 
corresponding to a maximum distance traveled by the light (a real-time trimap is 
generated for each frame based on the original depth matte, page 569, lines 22-23); 
and 

generating a depth mask identifying objects within a foreground region and a 
background region of the scene (foreground objects can be generated by setting the 
depth measurement window, page 568, section 3.2) based upon the light captured 
during the set time (a real-time trimap is generated for each frame based on original 
depth matte, page 569, 4th paragraph), the depth mask identifying objects within the 
foreground region with a first bit value (full value pixels representing foreground 
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pixels in the color channel) and identifying objects within the background region with 
a second bit value (zero value pixel representing background) (page 564, 6 th 
paragraph). 

Gvili does not specifically teach the adjusting image capture device parameters according 
to bit values of the depth mask for one or more of a sequence of captured image frames of the 
scene, wherein the image capture device parameters are selected from one of focus, 
brightness, exposure or gain. 

However Podoleanu does teach the adjusting image capture device (an optical mapping 
apparatus) parameters according to bit values of the depth mask for one or more of a 
sequence of captured image frames of the scene, wherein the image capture device 
parameters are selected from focus adjustment (Podoleanu teaches a method utilizes an 
optical mapping apparatus where the parameters of the said synchronising procedure of the 
optical coherence depth adjusting means and focusing adjusting means, parameters which are 
the range, initial position and velocity of the focus adjusting means for a given depth range 
and velocity of the optical coherence depth scanning means, wherein each optical coherence 
image in either loop is validated by the brightness and regularity of the confocal image or one 
of the confocal images) (see claim 73 and 32:66-33:32, Podoleanu). 

Thus, it would have been obvious to one of ordinary skill in the art to have included an 
optical mapping procedure as taught by Podoleanu into Gvili 's digital camera, as the 
Podoleanu procedure suggested an optical mapping algorithm with adjustable depth 
resolution and a multiple functionality to ensure that the confocal optical receiver operates at 
the maximum sensitivity (3:62-65, Podoleanu). 
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Gvili and Podoleanu do not specifically teach the adjusting of the image capture device 
parameters being done independently in the foreground from the background. 

However Kawaguchi does teach the adjusting of the image capture device parameters 
being done independently in the foreground from the background (the criteria and the 
conjugation operation Illustrate the pixels in the foreground area, e.g. B or Be, and pixels in 
the background area, e.g. W or Wc, of the image capture device can be adjusted 
independently) (3:66-4:31 and Figs 1-6, Kawaguchi). 

Thus, it would have been obvious to one of ordinary skill in the art to have included a 
main subject detection unit as taught by Kawaguchi into Gvili and Podoleanu's digital 
camera, as the Kawaguchi reference utilizes bit-plane complexity segmentation 
steganography as a technique for increasing information hiding capacity to a point where 
confidential human to human communications are possible, therefore provides an 
information hiding capacity of as much as 50% of the original image data and, in some 
circumstances, may provide hiding capacities in excess of 50% (2:12-19 and 5:26-37, 
Kawaguchi). 

Regarding claim 3, Gvili discloses the light source is configured to emit infrared light (IR 
laser diodes, page 567, lines 15-16). 

Regarding claim 4, Gvili discloses a method operation of opening an aperture cover 
allowing access to a sensor of the image capture device (the shutter is precisely controlling 
the exposure time of the CCD, page 567, lines 17-18) includes, receiving reflected light from 
the objects within the foreground region (light reflected from every object inside the depth 
measurement window, page 568, second paragraph of section 3.2). 
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Regarding claim 6, Gvili discloses a method operation of adjusting image capture device 
parameters according to bit values of the depth mask prior to capturing a subsequent 
corresponding image of the scene (ability to change the parameters of the depth window 
according to the scenario's need, page 568, section 3.2, first paragraph) includes, determining 
an optimal amount of light based upon the depth mask; and adjusting the aperture cover to 
allow the optimal amount of light into the image capture device (normalized depth of pixel 
can be calculated, page 567, lines 7-11). 

Regarding claim 7, Gvili discloses the image capture device parameters are adjusted 
through mechanical adjustments (depth key setting can be automatically set, page 568, third 
paragraph of section 3.2). 

Regarding claim 8, this claim recites same limitations as claim 3. Thus it is analyzed and 
rejected as previously discussed with respect to claim 3 above. 

Regarding claim 9, Gvili discloses a method for adjusting image capture settings for a 
single image capture device (a novel depth video camera), comprising: 

- identifying a scene (easily identified, page 569, third paragraph); 
capturing an image of the scene through the single image capture device (the 
depth information is captured by a camera, page 566, fifth paragraph); 
generating a depth mask of the scene from data defining the image of the scene 
(generating depth map according to some depth criterion, page 568, last 
paragraph); and 

- adjusting pixel values of the data defining the image corresponding to objects 
within any one or both of a foreground region and a background region of the 
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captured image (normalized depth of pixel D{i,j) can be calculated, page 567, 
lines 7-11) and the adjusting of pixel values is according to bit values of the depth 
mask (full value pixels representing foreground pixels in the color channel and 
zero value pixel representing background) (page 564, 6 th paragraph). 

Gvili does not specifically teach that adjusting pixel values associated with the 
foreground region are independent of adjusting of pixel values associated with the 
background region. However Kawaguchi does teach the adjusting pixel values associated 
with the foreground region are independent of adjusting of pixel values associated with the 
background region (the criteria and the conjugation operation Illustrate the pixels in the 
foreground area, e.g. B or Be, and pixels in the background area, e.g. W or Wc, of the image 
capture device can be adjusted independently) (3:66-4:31 and Figs 1-6, Kawaguchi). 

Thus, it would have been obvious to one of ordinary skill in the art to have included a 
main subject detection unit as taught by Kawaguchi into Gvili 's digital camera, as the 
Kawaguchi reference utilizes bit-plane complexity segmentation steganography as a 
technique for increasing information hiding capacity to a point where confidential human to 
human communications are possible, therefore provides an information hiding capacity of as 
much as 50% of the original image data and, in some circumstances, may provide hiding 
capacities in excess of 50% (2:12-19 and 5:26-37, Kawaguchi). 

Regarding claim 10, Gvili discloses the method operation of generating a depth mask of 
the scene from data defining the image of the scene includes segmenting the foreground and 
background regions of the scene (automatically segmentation of the scene is possible, page 
568, third paragraph of section 3.2). 
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Regarding claim 1 1 , Gvili discloses the data defining the image of the scene includes 
pixel data where each pixel is tagged with distance information (calculate color distances 
between neighboring pixels, page 570, section d). 

Regarding claim 13, Gvili discloses the image capture device is selected from the group 
consisting of a digital camera, a web cam, and a camcorder (a novel depth video camera, 
page 564, line 4). 

Regarding claim 14, Gvili discloses the displaying a portion of the image of the scene 
having adjusted pixel values (a new matte is generated by combing the information from both 
the depth and the color frames, page 569, fourth paragraph and also page 571, first paragraph 
of section 4). 

Regarding claim 15, Gvili discloses the portion of the image of the scene is an image of a 
participant for use in an interactive gaming application (man-machine interactions, page 572, 
second paragraph of section 5). 

Regarding claim 17, this claim differs from claim 9 only in that the claim 9 is a method 
claim whereas claim 17 is an apparatus. Thus the apparatus claim 17 is analyzed and rejected 
as previously discussed with respected to claim 9 above. 

Regarding claim 18, Gvili discloses the depth mask is a bit mask having a first logical 
value (full value pixels) assigned to represent the foreground objects and a second logical 
value (zero value pixels) assigned to represent the background objects (full value pixels 
representing foreground pixels in the color channel and zero value pixel representing 
background) (page 564, 6 th paragraph). 



Application/Control Number: 10/758,817 Page 9 

Art Unit: 2622 

Regarding claims 19, this claim recites same limitations as claim 4. Thus it is analyzed 
and rejected as previously discussed with respect to claim 4 above. 

Regarding claim 20, Gvili discloses each logic element is one or a combination of 
hardware (i.e. shooting lens, depth sensor, as well as a camera) and software (i.e. a depth 
map, depth measurement window, and scene segmentation) (pp. 568-569, section 3.2 depth 
key setting) 

Regarding claim 21, Gvili discloses the image capture device is a video capture device (a 
novel depth video camera, page 564, line 4). 

Regarding claim 22, Gvili discloses the depth logic is further configured to periodically 
provide a depth mask for a sequence of video frames captured by the video capture device 
(step f of the process for iteration of alpha values, page 570). 

Regarding claim 23, Gvili discloses the image capture device setting is adjusted through 
one of a mechanical or electrical adjustment (depth key setting can be automatically set, page 
568, third paragraph of section 3.2). 

Regarding claim 24, Gvili discloses the image capture logic is further configured to 
adjust each pixel of image data of the scene (step d of the process shown the color distance 
calculation is performed for each pixel of the image data of the scene)( step d of the process 
for iteration of alpha values, page 570). 

Regarding claim 25, this claim differs from claim 1 only in that the claim 9 is a method 
claim whereas claim 25 is an apparatus. Claim 25 further differs from claim 1 in that the 
limitations "a computing device" and "a display device" are additionally recited. Gvili 
teaches a computing device (a Pentium 3 machine, page 572, section 5) and a display (a 
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video camera, page 564, line 4). Thus the apparatus claim 25 is analyzed and rejected as 
previously discussed with respected to claim 1 above 

Regarding claim 26, Gvili discloses the computing device is a game console (game input 
device, page 572, section 5). 

Regarding claim 27 and 28, these claims recite same limitations as claims 22 and 23, 
respectively. Thus they are analyzed and rejected as previously discussed with respect to 
claims 22 and 23 above. 

Regarding claim 29, Gvili discloses the video capture device is a webcam (ZCam is used 
to generate depth keying during live broadcasts, page 568, line 5 and page 572, section 5). 

Regarding claims 30 and 32, these claims recite same limitations as claim 11. Thus they 
are analyzed and rejected as previously discussed with respect to claim 1 1 above. 

Regarding claim 31, Gvili discloses the scene image data includes an image of a person, 
the image of the person being incorporated into a video game for interaction therein (camera 
using depth-sensing technology are used in man-machine interactions) ( second paragraph of 
Summary, page 572). 

6. Claim 2 is rejected under 35 U.S.C. § 103(a) as being unpatentable over Gvili in view of 
Podoleanu and Kawaguchi, and further in view of Tuomi, US 7,061,507. 

Regarding claim 2, note the discussion of claim 1 above. Gvili, Podoleanu and 
Kawaguchi do not teach storing the depth mask in memory of the image capture device. 
However, Tuomi teaches storing the depth mask in memory of the image capture device 
(provided a Z-buffer 1902 for storing the Z- values relating to the depth of the pixel; see 
12:15-36, Tuomi). 
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It would have been obvious to one of ordinary skill in the art at the time this invention 
was made to have used a memory as taught by Tuomi as modified by Gvili, Podoleanu and 
Kawaguchi so that it provide multiple buffers for storing information (12:15-17, Tuomi). 



Conclusion 

7. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure: 

- Chang (US 2001/0017932) discloses a method for decoding a message embedded in a 
pattern of pixels, 

- Bender et al. (US 6,4 1 1 ,392) provides a technique for embedding a mark in an a 
printed image allows its interpretation by an inexpensive printing system, and 

- DeFreitas (US 5,870, 100) discloses a method for filling a selected region in a target 
image includes determining an attribute of a seed pixel within the selected region, 
forming a mask image of the region based on identifying pixels that have the 
attribute, and filling the selected region based on the mask image. 



Inquiries 

8. Any inquiry concerning this communication or earlier communications from the examiner 
should be directed to Kent Wang whose telephone number is 571-270-1703. The examiner 
can normally be reached on 8:00 A.M. - 5:30 PM (every other Friday off). 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Sinh Tran can be reached on 571-272-7564. The fax phone number for the 
organization where this application or proceeding is assigned is 571-270-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published 
applications may be obtained from either Private PAIR or Public PAIR. Status information 
for unpublished applications is available through Private PAIR only. For more information 
about the PAIR system, see http://portal.uspto.gov/external/portal/pair. Should you have 
questions on access to the Private PAIR system, contact the Electronic Business Center 
(EBC) at 866-217-9197 (toll-free)? If you would like assistance from a USPTO Customer 
Service Representative or access to the automated information system, call 800-786-9199 (IN 
USA OR CANADA) or 571-272-1000. 
/TuanVHo/ 

Primary Examiner, Art Unit 2622 
KW 

24 Dec 2008 



